Masking a snapshot without breaking foreign keys

A masked snapshot has two jobs that pull against each other. It has to be safe to hand to anyone with a preview link, which means no real names, emails or card numbers. And it has to behave like production, which means every join still joins, every unique index still holds, and the customer with 4,000 orders still has 4,000 orders. Most masking scripts manage one of the two. This is how database.mask in Canary does both.

Random replacement breaks things quietly

The obvious approach is to overwrite each sensitive value with a random one. It fails in three ways, and none of them is loud.

• Joins on natural keys. If invoices.billing_email matches users.email by value, two independent random strings no longer match. The rows are still there. The invoice page is just empty.
• Unique constraints. Random values collide eventually. With 48,210 users and a short list of fake names, the restore fails on the unique index, or worse, succeeds because someone dropped the index to make it pass.
• Stability. If a user gets a different fake email every night, yesterday's bug report ("log in as user_48210") points at someone else today.

Deterministic, per column, keyed per workspace

Canary replaces each masked value with a keyed hash of the original, formatted to fit the column. The same input always produces the same output inside your workspace, and a different output in anyone else's.

database:
  engine: postgres@16
  seed: snapshot://nightly
  mask: [users.email, users.name, cards.number]

With that config, a real address becomes user_48210@example.test everywhere it appears, tonight and every night after. Three properties follow.

1. Joins hold. Any column that stores the same value gets the same replacement, so a value-based reference still resolves. Canary reads your foreign keys and applies a parent column's rule to its children without being told. Columns that reference by value with no declared constraint need to be listed.
2. Uniqueness holds. The replacement is derived from the full original value, so two different inputs produce two different outputs. A unique index on users.email restores cleanly.
3. Shape holds. Emails stay valid emails on a reserved test domain. Names keep a plausible length. Card numbers keep their last four digits and still pass a checksum, so validation code does not reject the seed data.

The key lives in your runner and is never sent to us. Without it, the mapping cannot be reversed or reproduced.

Masking happens before the copy moves

The order of operations matters more than the algorithm. The nightly job runs at 2:00 AM PT inside the runner in your AWS or GCP account. It reads from a replica, applies the mask rules as rows stream out, and writes the snapshot to a bucket you own. There is no unmasked intermediate file. A preview restores from the masked snapshot and has no network path to the source database.

Nothing in that sequence touches Canary's control plane. We receive a status line: the snapshot's size, how long it took, and a count of rows per rule.

Four rules, not one

Every table ends up under one of four rules. The comment Canary leaves on the pull request lists them.

Tokenized is for values that have to work against a third party. A payment row keeps its amount and status, but the processor reference is swapped for a Stripe test token, so a refund in a preview exercises the real code path against the test API. Skipped is for tables that are large, sensitive and useless in review. The table exists, so migrations run, but it is empty.

JSON columns

Free-form JSON is where personal data hides. A mask rule can address a path inside a JSON or JSONB column:

mask:
  - users.email
  - users.profile.phone
  - events.payload.customer.email

The path is applied to every row where it exists and ignored where it does not. Arrays are walked. A path that matches nothing in a full night's snapshot is reported as a warning, because a typo in a mask rule should be loud. The changelog has the release notes.

What never gets copied

Some things are excluded whatever the config says: database roles and their passwords, replication slots, and extension-owned tables that hold credentials. Columns whose names match a short built-in list (password, secret, token, api_key and their common variants) are masked by default. You can turn that off per column, and when you do, the change shows up in the audit log with your name on it.

Masking is not a substitute for deciding what belongs in a preview. If a table has no business being reviewed, skip it. The smallest snapshot that makes the branch reviewable is also the fastest to restore: our 1.8 GB demo snapshot seeds in 48 seconds.

The key reference for canary.yml is in the docs, and the rest of what a preview isolates is on the features page.